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(57) i A composite camera system includes a control 
section for performing positional control and magnifica- 
tion ratio control of at least one zoom camera for omni- 
directional Image data centered on a prescribed portion 
thereof, the omnidirectional image data being obtained 



by an omhidlnectioftai camera capable of taking an om- 
nidin^onal'imag'e over a viewing angle of a maximum 
of 360 degrees; and a display section for displaying an 
omnidirectional image taken by the omnidirectional 
camera and a zoom image taken by the zoom camera. 
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Description 

BACKGROUND OF THE INVENTION 

1. FIELD OF THE INVENTION: 

[0001] The present invention relates to a composite 
camera system for taking an omnidirectional image us- 
ing an omnidirectional camera capable of capturing in- 
formation of a viewing angle of a maximum of 360 de- 
grees, and zoom-photographing an area including an 
image area which is automatically or manually specified 
from the orhnidirectioriaJ image by a zoom camera; a 
zoom camera Image display control method and a zoom 
camera control method using the ^me, a control pro- 
gram usable for the same,.and a cornputer readable re- 
cording medium having the progmm ^ storexl'thereih. 

2 DESCRIPTION OF THE RELATED ART 



[0002] Monitoring cameras conventionally and gener- 
ally known as used in security c^cei installed in, for 
example, financial institutions and supermarkets in- 
dudo fixed-type and pivotablc cameras Usually, a man- 
ager checks a video taken by such a rrioriftorihg camera 
using amonitoring device orthe like in the manager's 
office, so that the manager can recoggize any abnor- 
: mality caused by, for example; a t^passeror a fireand 
take emergency measures. 
[0003] A monitoring instrument for allowing the man- 
ager to monitor a video taken by the m onitoring camera 
on a screen of amonitoring device is inevitably large- 
sized and can be quite expensive. 'Even such enlarge 
instrument is still insufficient for accurately monitoring a 
target of crime prevention at a remote location. 
[0004] Generally known security monitoring devices 
include devices which issue an alarm using (i) sensors 
for sensing a trespasser who intercepts , for example, 
laser light or infrared rays; or (ii) sensors for sensing 
fires arid gas leaks. When such a security device using 
such a sensor gives an alarm, the manager can recog- 
nize that an abnormality has occurred, but cannot obtain 
accurate information by visually checking the degree or 
details of the abnormality. 

[0005] As the number of crimes increases year by 
year a security monitoring system has been demanded- 
for a'nowing the manager at a remote location to accu- 
rately identify the occurrence of an abnormality while : 
monitoring the target of crime prevention without requir- 
ing the manager to pay special attention. 
[0006] Proposals for solving the above-described 
problems are described in, for example, Japanese Laid- 
open Publication No. 3*4700 directed to -Remote 
Monitoring Device-,- Japanese Laid-Open Publication 
No 4-176484 directed to -Monitoring System in Play 
Hair, and Japanese LaidOpen Publication No. 
11-259775 directed to -Monitoring System".. 
[0007] The "Remote Monitoring Device- disclosed by 



Japanese Laid-Open Publication No. 3*4700 operates 
as follows. A plurality of sensors are dispersed in blocks 
of a monitoring area such that each sensor can detect 
- an abnormality to the respective block. When an abnor- 

5 mality is detected, aTV camera is automatically directed 
to the site of the abnormality, and transfers the abnor- 
mality detection signal and the video signal captured by 
the TV camera to a terminal in the monitoring office. The 
terminal automatically displays the image of the site of 

10 the abnormality and related information. Tnus, the situ- 
ation at the site of the abnormality is recognized easily 
and with certainty: 

(0008] The "Monitoring System i in Play Hail" disclosed 
by Japanese Laid-Open Publication No. 4-176484 op- 
is erates as follows A monitoring video camera is set to a 
monitoring state for monitoring a game machine, which 
has been determined to be in an abnormality based on 
the result of analysis of Information obtained by abnor- 
mality detection means. The video information on thai 
20 game machine is d^iayeci on the screen of the video 
. cajriera. Thus, the. monitoring efficiency of each game 
machine can be irinprbved at low equipment cost. 
[0009] Tbe -Monitoring System" disclosed by Japa- 
nese Laid-Open Publication No. 11-259775 operates as 
25 follows. An image of a monitoring area is captured by a 
wide^range camera, and the captured image is com- 
pared with an image of a normal state so as to find a 
■ d&erence. tnus, ah abnormality is detected. When en 
abnormalrty occurs, an image of the abnormality is cap- 
30 tured by the wide-range camera, and also an image at 
an important location is captured by a zoom camera. 
The Image obtained by the wide-range carneraandthe 
image obtained by the zoom camera are transferred to 
. a terminal computer. -Thus, a manager at a remote lo- 
ss cationcaruecognizeth^ 

relatively in detail, and accurately specify the degree of 
the abnormality and the like. 
[0010] The above-described technologies have the 
following problems: 
40 [0011] In the case of the "Remote Monitoring Device- 
disclosed by Japanese Laid-Open Publication No. 

3- 84700, the sensors are dispersed. This means that 
tn'e sensors can detect abnorm alities only at sites where 
the sensors are located/In addition, certain types of sen- 

45 . sor can only detect -an abnormality of the respective 
. . type. There are other problems in that the reliability of 
the sensors dispersed may be lowered, and it is difficult 
to control the posrtion of the TV camera with respect to 
the sensors. 

so poi2] in the case of the -Monitoring System in Play 
- s Hall* disclosed by Japanese Laid-Open Publication No. 

4- 176484, abnormality detection means including vari- 
ous sensors needs to be located in the vicinity of each 
game machine: Therefore, abnormalities only at sites 

55 where the sensors are located can be detected. There 
are other problems in that the sensors may malfunction 
because of the environment in which the sensors are 
located, arid it is troublesome to control the position of 



3 EP 1 34 

the TV camera with respect to the various sensors. • . . 
[0013]' In the case of the •Monitoring System 0 dis- 
closed by . Japanese Lafd-Open . Publication No. 
1 1 -259775,:the wide-range camera acts as- one- sensor 
f or comparing an Image of a target of monitoring and an 
image of a normal state -and detectingan abnormality 
based on the difference. The zoom camera is set.to.be 
directed to several p reset important locations . The zoom 
camera is not structured so as to track the sites of ab- 
normality photographed by the wide-range camera and 
to have the position and the magnification ratio thereof 
controlled. Therefore, when there are many important 
locations, the important location at which an abnormality 
has occurred needs -to be visually checked. This re- 
quires images of a plurality of important locations to be 
taken simultaneously. 

SUMMARY OF THE INVENTION : 

[0014] According to one aspect of the . invention, a 
composite camera system includes a control section for 
performing positional control . and magnification ratio 
control of at least one zoom camera ratio for omnidirec- 
tional imago data, centered on ; a prescribed portion 
thereof, the omnidirectional image. data being obtained 
by, an omnidirectional camera capable of taking an om- 
nidirectional image . over a viewing angle of a maximum 
of 360 degrees; and a display section for displaying an 
omnidirectional image taken by . the -omnidirectional 
camera and a zoom image taken by the zoom camera; 
[0015] :ln one embodiment of the invention, the . posi- 
tional control for controlling a photographic position is 
performed by pan driving and tilt driving; and the rriac~ 
nification ratio control is- performed by. zoom driving. 
[00T6] . In one embodiment of the invention, the pre- 
scribed portion of the-.omnidirectional image data is a 
divided image area which is set in a photographed inv 
age: .-■ .... 

[0017] . In one embodiment of the invention, the control 
section includes a dividing section for dividing the on> . 
nidirectional image displayed on a display screen.of the 
display section into a. plurality of image areas; a speci- 
fying section for automatically or manually specifying at 
least one image area from the plurality of image areas; 
a converting section for converting Information on the at 
least one image area specified by the specifying section . 
into control information for controlling the zoom camera 
to be at a prescribed position and to have a prescribed 
magnification ratio, for photographing ■ arijrhage area 
specified by the specifying section; and a zoom camera 
control section for controlling the zoom camera to be at 
the prescribed position.and to have the prescribed mag- 
nification ratio for photographing the image area speci- 
fied by the specifying section.ibased On the control in- 
formationconverted by the converting section. 
[0018] - In one embodiment of the invention, the omni- 
directional camera includes an- optical system for pro- 
jecting the omnidirectional image using a mirror of a par- 
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abolic or hyperbolic convex body of revolution; or a fish- 
eye lens. 

[0019] In one embodiment of the invention, the com- 
posite-camera system further includes a control infor- 

5 mation setting section for presetting the control informa- 
tion for controlling Jhe zoom camera to be at the pre- 
scribed position and to have the prescribed magriifica- 
. rjon ratio for each of the plu rality of image-areas obtained 
by the dividing section; and a control information table 

10 for storing the preset control information in correspond- 
ence with the position of each of the plurality of image 
areas. The converting section obtains the control infor- 
mation for the image area specified by the specifying 
section from the control information table and/or from 

15 calculation using a prescribed calculation procedure. 
. [0020] In one embodiment of the invention, the com- 
posite camera system further includes an Image mem- 
ory section. for temporarily storing the omnidirectional 
image taken by the omnidirectional camera frame by 

20 frame sequentially; and a moving body detection section 
for detecting a movement of a moving body by perform- 
. ing pattern matching of successive frames of the omni- 
directional image stored in the image memory section. 
The detected moving body Is zoom-photographed by 

25 the zoom camera; and an enlarged image of the moving 
body is displayed oh the display screen of the display 
■ section..- 

: [0021] - In one embodiment of the invention, when the 
omnidirectional image is circular, the dividing section di- 

30 vides the omnidirectional image concentrically and ra- 
dially, -and displays the plurality of image areas of the 
omnidirectional image together with area frames defin- 
ing me plurality of image areas.- 
[0022] In one embodiment of the invention, the divld- 

35 ing section displays the plurality of image areas of the 
omnidirectional image together with area frames and ar- 
ea numbers defining the plurality of image areas . The 
control information setting section presets the control in-" 
. formation for controlling the zoom camera to be at the 

*o prescribed position and to have the prescribed magnifi- 
cation ratio for each of the plurality of image areas pro- 
vided with the area numbers. - . 
[0023] In one embodiment of the invention, the com- 
posite camera system further includes an invalid area 

45 setting section for presetting, as invalid, an arbitrary ar^ 
ea of the omnldlrectlonal lmage which 1 is unstable due 
to envirbnrriental influence. The invalid area setting sec- 
tion invalidates the specification of the image area by 
the specifying section. 

so [0024] In one embodiment of the invention, the com- 
posite camera system further includes a monitoring area 
setting section forsettihg at least one monitoring area 
jnthe omnidirectional image; and an alarm information 
output section f or outputting alarm information when the 

55 moving body detecting section detects that the moving 
body has entered the monitoring area- The zoom cam- 
era zoom-photographs the moving body which has en- 
tered the monitoring • area and displays the photo- 
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graphed image of the moving body on the display screen 
of the display section. 

[0025] In one :erribodiment of the invention, the com- 
posite camera system further Includes a remote control- 
ler connected to the omnidirectional camera :and the s 
zoom camera v ia a wireless .or wired communication 
line; An omnidirectional image taken by the omnidirec- 
tional camera and an image taken by the zoom camera 
are transferred to" the remote controller via the commu- 
nication line. ".- « . > . 
[0026] In one embodiment of the invention, the n> 
mote controller includes at least the dividing section - the 
specifying section. and the converting section.- . 
[0027] According to another aspect of the invention, 
a composite camera -image display control method in- 
cludes the processing steps of performing positional 
control and magnification ratio control of at least one 
zoom camera for omnidirectional :image : data centered 
on a prescribed portion thereof, the omnidirectional im- 
age data being obtained by ah omnidirectional camera 
capable of taking an omnidBrectiohal image over a view- 
ing angle of a maximum of 360 degrees; and displaying 
an omnidirectional image taken -by the . omnidirectional 
camera and a zoom Image taken by the zoom camera. 
[0028] According to still another aspect of the inven- 
tion, a zoom camera control method for performing po- 
sitional control and magnification ratio control of at least 
one zoom camera for omnidirectional image data : cen- 
tered on a prescribed portion thereof is. provided. The 
omnidirectional imagedata is obtained by an omnidirec- 
tional camera capable of taking ah omnidirectional im- 
age over a viewing angle of a maximum of 360 decrees. 
The method includes the steps of displaying an omnidi- 
rectional image taken by the omnidirectional camera; di- 
viding the omnidirectional image displayed on a display 
screen of the display section into a.plurality of image 
areas ; automatically or man ualry specifying at least one 
image area from the plurality of image areas; coriveTting 
information on the at least one image area specifiedby 
the specifying section into . control' information for con- 
trolling the zoom camera . to be at 8 [ prescribed position 
and to have a prescribed magnification ratio for photo- 
graphing the image area specifiedby thespedfyingiec- 
tion; and controlling the zoom camerato be at the pre^ 
scribed position arid to have the prescribed magnifica- 
tion ratio for photographing the Image area-specified by 
the specifying section; based on thecbntrol information : 
converted by the converting section. : 
[0029] According to still another aspect of the invert 
tion, a control program is provided. for causing a com- 
puter to execute the steps of displaying an omnidirec- 
tional imago taken by tho omnidirectional camera; divid- 
ing the omnidirectional image displayed on a display 
sicreen of the display section into a plurality of image 
areas; automatically or manually specifying at least one 
image area from the plurality of imageareas; converting 
information on the at least one image area specified by 
the specifying section into control information for con- 



trolling the zoomxamera to be at a prescribed position 
and to have.a preserved magnification ratio for photo- 
graphing the image area specified by the specifying sec- 
tion; and controlling the zoom camera to be at the pre- 
scribed position and to have the prescribed magnifica- 
tion ratio for photographing the image area specified by 
the specifying section, based on the control information 
converted by the converting section. 
[0030] . According to still another aspect of the invent 
tion, a computer readable recording medium having the 
above described control program stored therein is pro- 
vided. 

[0031] According to still another aspect of the inven- 
tion;: a control program for causing a computer to exe- 
cme the processing steps of the above-described com- 
posite camera image display control method is provid- 
ed. 

[0032] As described above; according to the present 
invention, positional control and magnification ratio con- 
trol of the zoom camera are performed centered on a 
prescribed portion of omnidirectional image data; and 
the omnidirectional image and the zoom image are dis- 
played. The omhidirectional camera is used to photo- 
graph a plurality of important locations simultaneously 
as included in the omnidirectional image, and the zoom 
camera is used to zoom -photograph a specified image 
area in the omnidirectional image. By specifying a pre- 
scribed image area in the omnidirectional image, the po- 
sition of the zoom camera can be easily set. 
[0033] The omhidirectioh aT image "over a viewing an- 
gle of a maximum of 360 degrees taken by the omnidi- 
rectional camera is divided into a plurality of image ar- 
eas,: and an image area of interest -is automatically or 
manually specified among the plurality of image areas. 
Thus* the specified inriage area of interest is zoom^pho- 
tographed by the zoom camera which can be driven to 
pan, lift and zoom. Since a large-area monitoring area 
is photographed by one omnidirectional camera, it is not 
necessary to install a plurality of monitoring sensors , 
for example; a plurality of detecting sensors or Input sen- 
sors for automatic specification of the target image area. 
[0034] The plurality of image areas obtained by divid- 
ing the omnidirectional image act as a plurality of detec- 
tion sensora br.input sensors to calculate the control In- 
formation for the image area automatically or manually 
specrfied. 'Based oh^the control Information,: the zoom 
ccirnera is corrtrblted to be at a prescribed position (to 
assume a prescribed photographing direction) and to 
have' a prescribed magnification ratio. • 
[0035] The zoom camera can track a moving body in- 
.cluded in the omnidirectional image taken by the omni- 
directional camera, zoom-photograph the moving body, 
and display an enlarged image of the moving body on 
a display screen of the display section. 
[0036] In the case wherea circular omnidirectional im- 
age is divided into a plurality of image areas, and control 
information is set for controlling the -zoom camera to be 
at a prescribed position and to have a prescribed mag- 
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nification ratio for each of the plurality ; of image areas, 
the division can be performed effectively andthe image 
areas can be. displayed so as to be easily identifiable. 
[0037] Control Information for : controlling the zoom 
camera tobe at a prescribed positioned to have a pre- 
scribed magnification ratio for each of the plurality of im- 
age areas provided with the respective area numbers 
can be preset, while viewing the omnicfirectional image 
displayed on the display screen of the display section. 
[0038] An image area, in the omnidirectional image, 
which is unstable due to environmental influence can be 
invalidated in advance. Therefore,a moving body or an 
abnormality can be accurately detected automatically 
from the omnidirectional image, so that the position and 
the magnification ratio of the zoom camera is accurately 
controlled for the detected moving body orthe site of the 
abnormality. 

[0039] Aplurality of monitoring areas which are to be 
photographed in different photographing directions can 
be set for one omnidirectional camera. When a moving 
body is detected to have entered the monitoring area, 
alarm information including an alarm sound is output 
and also the zoom camera is zoomed to the moving 
body to display the .image of themoving body on the 
display screen of th e display section: 
[0040] The omnidirectional camera and the zoom 
camera can be remote-controlled via a communication 
line. Therefore, at a site where a computer is installed, 
the above-described division of the omnidirectional im- 
age; preset of the control information for controlling the 
position and the magnification ratio of the zoom camera 
for a specified image area, and photographing and dis- 
play of an area including the specified image area can 
be performed. : : - : - ;* ■ 

[0041] Thus, -the invention described, herein makes 
possible the advantages of providing a composite cam- 
era system for photographing a plurality of important lo- 
cations simultaneously as included in an omnidirection- 
al image using an omnidirectional camera, and zoom- 
photographing a specified image area in the omnidirec- 
tional image with the position of a zoom camera being 
easily controlled; a zoom camera image-display control 
method.and a zoom camera control method using the 
same; -a control program usable -for: the same; . and a 
computer readable recording medium i . having the pro- 
gram stored therein. 

[0042] .These and other advantages of the present in- 
vention will become apparent to those skilled in the art 
upon reading and understanding the following detailed 
description with reference to the accompanying figures. 

BRIEF DESCRIPTION OF THE DRAWINGS . 

[0043] 

• • Figure 1 is a schematic : view illustrating a basic 
structure of a composite camera system according 
to an example of the present invention; 



• ■ Figure 2 shows an example of a monitoring system 
. using the composite camera system shown in Fig- 

' ure 1; • 

5 Figure 3 is a block diagram illustrating a partial 
. . . structure of the composite camera system shown in 
Figure 1; . . 

Figure 4 is a perspective view illustrating a partial 
10 structure of an optical system of an omnidirectional 
camera included in the composite camera system 
shown in Figure 1; 

, Figure -5 shows Display Example (1) according to 
: is the example of the present invention, which is tor 
setting control information; 

-= Figure 6 shows Display Example (2) according to 
: the example of the present invention, which is for 
20 . setting control information; 



Figure 7.shows Display Example ; (3) according to 
.. the example of the present invention, which is for 
setting control information; 

Figure 8 shows Display Example (4) according to 
- the example of the present invention , which is for 
detecting a moving body; 

30 . . Figure 9 shows Display Example (5) according to 
the example. of the present invention, which is for 
setting an invalid area; and . 

. Figure 1 0 is a flowchart illustrating a basic operation 
35 • procedure of the composite camera-system shown 
in Figure 1/ : 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS : . 

[0044] Hereinafter, the present invention will be de- 
scribed by way of illustrative, but non-limiting examples 
with reference to the accompanying drawings: . 
[0045] Figure 1 is a schematic view illustrating a basic 

45 structure of a composite camera system 10 according 
to an exarnple of the present invention/: 
[0046] In Figure 1, the composite camera system 10 
includes a zoom camera 1i an omnidirectional camera 
2, a. remote controller 3; a zoom camera control unit 4 

so acting as a zoom cameraxontrol section, a monitor 5, 
and communication lines 6a through 6d for connecting 
these elements so that those elements can communi- 
cate with each other. 

[0047] The zoom camera 1 includes; for example, a 
S5 zoom lens, an imaging section (photographing section) 
including : CCDs,;and a driving section for driving and 
controlling the zoom lens to pan (to move to a prescribed 
position in a horizontal direction), to tilt (to move to a 
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prescribed position in a vertical direction) and to zoom 
(to have, a prescribed magnification ratio). The zoom 
camera 1 also includes a communication section for 
transferring a control signal or an image signal via a 
wireless or wired line;. Based on control information sent 
f rom the zoom camera control unit 4, the zoom camera 
1 is driven and controlled to be a prescribed position (by 
pan and tilt) and to have a prescribed magnification ratio 
(by zoom). For example: 

zoom ratio:. . optical zoom: X23; ... 

electronic zoom: X10 
zoom speed: MAX1 .6 seconds (at the preset time) 

number of presets: 255.(255 positions 

and magnification ratios) 

[0048] The omnidirectional camera 2 includes, for ex- 
ample.an optical system <e;g. . a mirror of aconvex body 
of revolution, or a fish-eye lens) for projecting an optical 
image of an area over a viewing angle of a maximum of 
360 degrees, and an imaging section (photographing 
section), including CCDs, for photographing the optical 
image as a circular.omnidirectionarimage. For example, 
a detection area (monitoring area) can have a radius of 
15 meters from the omnidirectional camera 2. 
[O049] The remote controller 3 includes, for example, 
a personalcomputer which can communicate with other 
devices via a wireless or wired line; The remote control- 
ler 3 includes an image area dividing section (dividing 
section) for dividing an omnidirectional image taken by 
the omnidirectional camera 2 into a plurality of image . 
areas; ah image area specifying section (specifying sec- 
tion) for automatically or manually specifying at least 
one irriagearea from the : plurality of image areas ob- 
tained by the image area dividing section:. a control in- 
formation converting section (converting section) for 
converting the information on the image area specified 
by the specifying section info control inf drmatidn fbrcon- 
trolling the zoom camera 1 to move to a prescribed po- 
sition (by pan driving arid tilt driving) and to have a pre- 
scribed magriif icatibn ratio (by zoom driving) for photo- 
graphing the image area specified by the image area 
specifying section; and a display section for displaying 
the- omnidirectional image taken by the omnidirectional 
camera 2 on a display screen. : , 
[0050] The zoom camera contra unit 4 includes a 
computer:including, for example, a CPU, : ROM, RAM, 
and I/O port. Based on the control information supplied 
from the remote.controller 3,- the zoom camera control 
unit 4 controls the position and. the maignification ratio 
of the zoom camera 1 . The zoom camera control unit 4 
has a function of communicating with other devices via 
a wireless or wired line. 

[0051] The monitor 5 includes, for example, a liquid 
crystal display (LCD), a plasma display (PD)ror an elec- 
troluminescence display (ELD). Themonitor 5 displays 
an image which is zoom-photographed by the zoom 
camera 1 . ; " '* 



[0052] Among the communication lines 6a through 
6d, the communication fine 6a communicates the omni- 
directional camera 2 and the remote controller 3 to each 
other by an NTSC system; :The communication line 6a 
5 " transfers an omnidirectional image taken by the omni- 
directional carnera 2 to the remote controller 3. The 
communication line 6b connects the remote controller 

3 and the zoom camera control unit 4 to each other by 
an-' RS-232C system; The communication line 6b 

10 transfers , to the zoom camera control unit 4. control in- 
•formation for controlling the zoom camera 1 to be at a 
prescribed position (to assume a prescribed photo- 
graphing direction) and to have a prescribed magnifica- 
tion ratio. The communication line 6c connects the zoom 

is camera control unit 4 and the zoom camera 1 to each 
other by an RS-485 system, so that the zoom camera 
control unit 4 controls the position and the magnification 
ratio of the zoom camera 1 for zooming the specified 
image area. -The communication. line 6d connects the 

.20 zoom camera 1 and the monitor 5 by an NTSC system. 
The communication line 6d transfers' an image zoom- 
photographed by the zoom camera 1 to the monitor 5. 
[0053] In this examplei the zoom camera control unit 

4 is provided as a separate unit. Alternatively, the zoom 
25 camera control unit 4 .may be incorporated into the re- 
mote controller3 or the zoom camera 1 : The zoomcarti- 
era 1 and the omn (direction aJ camera 2 can each have 
a function of communicating with other devices via a 
wireless or wired line: 

M [0054] Figure 2 shows an exemplary monitoring sys- 
tem including the composite camera system 10 shown 
in Figure .1 . Identical elements previously discussed with 
respect to Figure T bear Identical reference numerals 
arid the detailed descriptions thereof will be omitted. 

35 [0055] In Figure 2, the composite camera system 10 
continuously monitors-sites for monitoring such as, for 
example, a schoolyard/school gate/gym/elevator en- 
trance,^ parking lot, an' office; a store, an apartment 
building, or a place of amusement, with the omnidirec- 

40 tion al camera 2 over a wide range (viewing angle: 360 
idegrees) with no . dead angle. When detecting a tres- 
passer M, the zoom camera 1 is controlled to be directed 
: to the .trespasser M and zoom-photographs the tres- 
passer M at a prescribed magnification ratio- The image 

45 ^displayed by the monitor 5 /which islocated separate- 
ly from the remote controller 3 but acts as a display sec- 
tion of.the.rerriote controller 3. An enlarged image of the 
. trespasser M may be displayed both on a display screen 
of the remote controller 3 and a display screen of the 

so monitors.- 

[0056] In this example, the zoom camera 1 and the 
omnidirecfiorial camera 2 are installed over the entrance 
to a school as shown in Figure 2. The trespasser M en- 
tering through the school gate is detected by the omni- 

55 directional camera 2 and is zoom-photographed by the 
zoom camera 1 -. An enlarged image of the trespasser M 
is displayed on the monitor 5 installed in a monitoring 
room. 
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[0057] Although, not. shown in Figure .2, the remote 
controller .3 .for remote-controlling :the omnidirectional 
camera 2, the zoom camera control unit 4 for controlling 
the position and the magnification ratio of the zoom cam- 
era 1; arid the communication lines' 6a through 6d for 
communicating theremote controller 3,the'zoom cam- 
era 1, the omnidirectional camera 2 and thelike are eJso 
installed in the monitoring room. 
;[0058] Figure 3 is a block diagram illustrating a partial 
structureof the composite camera system 10 shown in 
Figure 1 .".Identical elements previously discussed with 
respect to Figure 1 bear identical reference numerals 
and the detailed descriptions thereof .will be omitted. 
[0059] In Figure 3, as described above, the composite 
camera system 10 includes the zoom camera 1 , the orri^ 
nldirectional camera 2, the remote controller 3, the zoom 
camera control unit 4 acting as a zoom camera control 
section,, the monitor 5,- and communication lines 6 (6a 
through 6d as shown in Figure 1). The composite cam- 
era system 10 further includes analarm information oiit : 
put device 7. •= . . * - : 
[0060] The zoom camera 1 includes , for example, an 
imaging section (photographing section) 1a; a driving 
section 1 b for driving and controlling - the zoom lens to 
pan (to move to a prescribed position in a horizontal di- 
rection), to tilt (to move to a prescribed position in aver- . 
tical direction) and to zoom (to have a prescribed rriacf 
nification* ratio);, and -a communication section 1c for 
communicating with, other ; devices via a .wireless or 
wired line fortransf erence of control signals and image " 
signals/ Based on control information sent from the 
zoom camera control unit 4 , the zoom camera 1 is con- 
trolled to have a prescribed position- (to assumed i pre: 
scrtoed photographing -direction) :and to . have . a. pre- 
scribed magnification ratio; The driving section 1b in- 
cludes, for example, a- pulse motors DC servo motor, : 
anultrasoriicmdtor, or an encoder, and has a function of 
automatic high-speed three-dimensional control of the 
positionand the magnification ratio. . . 
[0061] . As shown in Figure 3, the omnidirectional cam- 
era 2 includes an optical system 2a, an imaging section 
(photographic section) 2b including fcCDs, and a com- 
munication section 2c for.communicating with other 
vices for transference of control signals and image sig- 
nals via a wireless or wired line. . - ■ 
[0062] The optical system 2a Includes, for example, 
a parabolic or hyperbolic mirror of a convex body of rev- 
olution .or a, fish-eye : lens, and a. cylindrical -or cup- 
shaped light-transmissive body for holding a CCD cam- 
era which is'iriduded in the imaging'section 2b. The op- 
ticai system : 2a projects ah optical image overman omni- 
directional viewing engieof a maximum of 360 degrees. 
For example, -the optical system 2a projects an optical 
image obtained by projecting (collecting) light reflected 
by the mirror of a convex body of revolution by a lens or 
a fish-eye lens. . . : .- ■ 

[0063] The imaging section 2b includes a CCD cam- 
era including, for example,:an imaging lens, a CCD sec- 



tion, an A/D conversion circuit, and an image processing 
circuit. The Imaging section 2b photographs the optical 
image-projected by the optical system 2a as omnidirec- 
tional image data. 

5 [0064] The zoom-camera control unit 4 includes a 
computer including,: for example; a CPU, ROM, RAM , 
and I/O port: The zoom camera control unit 4 also has 
a function of communicating ;with f other devices via a 
wireless or wired line; Based on the control information 

10 supplied frorrithe remote controiler3,the zoomcamera 
control unit 4 controls the position and the magnification 
ratio of the zoom camera 1? . 
[0065] The monitor 5 includes a liquid crystal display 
(LCD), a plasma display (PD), an electroluminescence 

is display (ELD), or various other display devices. The 
monitors also has a function of communicating with oth- 
er devices via a wireless or wired line; The monitor 5 
receives ; ah. enlarged Image which is zoom-photo- 
graphed by the zoom camera 1 from the remote control- 

20 ler 3 and displays the -enlarged image on a display 
screen thereof. - . • 

[0066] The communication lines' 6 include a plurality 
of communication lines for transferring data signals by, 
for cxamplo. an NTSC, RS-232C; or RS-4B5 system. 

25 The communication K lines 6 may use a wireless LAN for 
communicating specific short-distance aerial transmis- 
sion signals such as; for exarhple, ;1rDAControl-system 
Infrared signals orBliietooth-system radio wave signals. 
[0067] : The alarm information output device 7 in- 

30 eludes, for example; a buzzer, speaker, and an alarm 
lamp. When/for example; a moving body such as a tres- 
. passer M (Figure 2) enters a monitoring area, the alarm 
informatiomoutput device 7 outputs alarm information 
including an alarming sound and an alarming" light . ' 

35 [0068] ; The remote controller s includes a control sec- 
tion 3a, a program memory 3b, a buffer memory 3c, an 
image memory section 3d, a display section 3e?acon- 
troi information table 3f, an operation input device 3g,.a 
communication section 3h, and a bus 31 for transferring 

40 data between these sections. 
. [0069] ' The Control section 3a Includes a CPU (central 
processing unit) of the compiiter. Based on a control 
program, the control section 3a controte the sections of 
the composite camera system 10. The detailed structure 

45 of the control section 3a will be described later. 
. [0070] The program .memory 3b includes; for exanv 
pie, a readable recording" medium /such as a ROM; 
EPROM. EEPROM, Floppy® disc; optical disc (for ex- 
ample,. compact disc); . or hard disc. The program mern- 

50 pry 3b stores the control program for starting the control 
. section 3a and causing the control section 3a to execute 
• various operations:. { ■ 
[0071] The buffer memory 3c Includes, for example, 
- a readable recording medium such as a RAM, EPROM, 

55 ' EEPROM/:Floppy® disc or hard disc. The buffer mem- 
ory 3c temporarily stores data which is being controlled. 
[0072] The image memory section 3d includes, for ex- 
ample, -.a* readable recording medium such as a RAM, 
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EPROM. EEPROM, Floppy® disc or tmrd disc. The im- 
age memory section 3d temporarily stores an omnidi- 
rectional image taken by the omnidirectional camera 2 
frame by frame sequentially. . 
[0073] . The display section 3e Includes, for example, 
a liquid crystal display (LCD); .a plasma display (PD),.or 
an electroluminescence display (ELD);The display sec- 
tion 3e displays an omnidirectional image taken by the 
omnidirectional camera 2 on a display screen thereof. 
[0074] The control information table 3f includes, for 
example^ aTeadable.recording medium such as a RAM, 
EPROM, EEPROM, Floppy® disc or hard disc; The con- 
trol information table 3f stores control information (infor- 
mation on the position (photographic direction) and the 
magnification ratio of the zoom camera 1 ),which is pre- 
set by a controljnformation setting section 33a (de- 
scrlbedbelow) Included in the control section 3a; in cor- 
respondence with Information on each of the image ar- 
eas (for example, the area numberor the representative 
coordinate of each of the image areas).. 
[0075] The operation input device 3g includes, for ex- 
ample, a mouse and a keyboard. .. 
[0076] : The communication section 3h includes, for 
example, an antenna; modem (sighalmodulation and 
demodulation device), a wireless.signal conversion cir- 
cuit, -or a communication line connection circuit. The 
communication section 3h is communicable with the 
omnidirectional camera 2, the zoom camera control una 
4, .the monitor 5, the alarm information output device ? 
and the like via the communication lines 4>; 
[0077] The detailed structure of.the control section 3a 
of the remotecontroller.3 will be described. The control . 
section 3a includes a dividing section 31a, a specifying 
section 32a, the controrinfofmatibn setting section 33a; 
a converting section 34a, a moving body detecting sec- 
tion 35a; an invalid area setting section 36a -a monitor- 
ing area setting section 37a, and a cursor control section 
38a. The control section 3a drives the zoom camera 1 
through the zoom camera control unit 4 so as to be at a 
prescribed position (by pan and tilt driving) and to' have 
a prescribed magnification ratio (by zoom driving) tor 
photographing theomritdirectional image data obtained 
.by the omnidirectional camera 2, centered on a speci- 
fied portion thereof . ; ■■ ■ 
[0078] - Based on a dividing program of the control pro- 
gram; the dividing section 31 a divides an omnidirection- 
al image, for example, concentrically and radially. Then, 
the dividing section 31a displays the divided image ar : 
eas of the omnidirectional image, together with an area 
frame which clearly defines each: divided imap^ . area 
and an area number added to each divided image area. . 
[0079] Based on an operation specifying instruction 
fromthe operation input device 3g and a specifying pro- 
gram of the control program* tine specifying section 32a 
automatically or manually specifies an image area (or a 
prescribed number of image areas) defined by an area 
frame (or a prescribed number of image frames) and an 
area number (or a prescribed number of area numbers), 



among the plurality of image areas displayed. 
[0080] Based on an operation setting instruction from 
the operation input device 3g and aoontrol information 
setting program of the control program, the control in- 
formation setting section 33a . presets control ..informa- 
tion for controlling the position and the magnification ra- 
tio of the zoom camera 1 on an image area-by-image 
area basis. 

|0081] Based on a conversion program of the control 
program, the converting section 34a converts informa- 
tion on the image area specified by the specifying sec- 
tion 32a into control information for controlling the posi- 
tion and the magnification ratio of the zoom camera 1 
for photographing theimage area specified by the spec- 
ifying section 32a. The converting section 34a may ob- 
tain the controljnformation for the specified image area 
from the control inforrhation table 3f and/or from calcu- 
lation using a prescribed calculation procedure, based 
on the conversion program . 

[0082] .Based on a moving body detecting program of 
the control program, the moving body detecting section 
35a performs pattern matching of successive frames of 
the omnidirectional image which has been stored in the 
image memory section 3d, so as to create a difference 
image. Based on the difference image, the moving body 
detecting section 35a detects.a movement of a moving 
body (including, for example, an object which changes 
the irtiage.'.such as a trespasser M shown in Figure 2). 
The: detected moving body is zoom-photographed by 
the zoom camera 1 , and an enlarged Image of the de- 
tected moving body is displayedon a display screen of 
the monitor 5 and/or the display section 3e. . 
. [0083] Based on an operation setting instruction from 
the operation input device 3g and an invalid area setting 
program of the control pr6gram t the invalid area setting 
section 36a presets; as Invalid, an arbitrary image area 
in the omnidirectional image which is unstable; due to 
environmental influence;. The -expression "image area 
which is unstable: due to -environmental influence" 
means an image area representing; for example, an ob- 
ject which moves in. a honnhal state^or an object which 
moves but . is too small to be a trespasser or anything 
causing an abnormality. The invalid area setting section 
36a can also display the image area which has been 
preset as invalid on the display section 3e, so that the 
image area Is identified. When the image preset as 
invalid has been specified by the specifying section 32a, 
the invalid area setting section 36a oan invalidate the 
specification;- 

. [0084]' . Based on an operation setting instruction from 
. the operation input device 3g and a monitoring area set- 
ting program of the control program, the monitoring area 
setting section 37a sets at least one monitoring area in 
the omnidirectional area. Any image area other than the 
image area which has been .invalidated by the invalid 
area setting section 36a may be set as a monitoring ar- 
ea/ • - ,:*" - ..*■- -• 
[0085] Based on an operation instruction from the op- 
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eratiori input device 3g and a cursor control program of 
the control program, the cursor control section 38a con- 
trols the position of the cursor on the display screen. 
{0086] The composite camera system 1 0 in this ex- 
ample having the above^described structure; when ap- 
plied to a monitoring system, provides, for example, the 
following effects. - 

(1) The omnidirectional camera 2 realizes monitor- 
ing over 360 degrees- with no dead angle. . 

(2) A: combination of the omnidirectional camera 2 
and the usual zoom camera Trealizes high-speed 

- display of a high-quality zoom-up image. 

(3) An' automatic tracking function is provided of de- 
tecting a moving body by the omnidirectional cam- 
era 2 and zoom-photographing the moving body by 
the usual zoom camera 1 . : 

. (4) The omnidirectional camera 2 realizes wide-ar- 
ea detection. .•<...•'.■.■ 

(5) A manual tracking function is provided of zoom- 
photographing.at ahigh speed, an omnidirectional 
. image, centered on a specified area thereof. 

[0087] The above-described control .program is 
stored in prescribed readable-recording memories in the 
remote controller .3 as follows.- 
[0088] The program memory 3b stores a display step 
of displaying an omnidirectional image taken by the om- 
nidirectional camera 2 on a.display -screen; a dividing 
step of dividing the omnidirectional image into a plurality 
of image areas; a specifying step of automatically or 
manually specifying at.leasj one image area from the 
plurality of divided image areas; and a converting step 
of converting inf ormation on the specified image area 
into control information for controlling the zoom camera 
1 to be at a prescribed position and to have a prescribed 
magnification ratio ;for the specified image area. . 
[0089] . A memory section in the zoom camera control 
unit 4 stores a zoom camera control step for controlling 
the position and the magnification ratio of the zoom cam- 
era 1 for photographing the image area specified by the 
specifying section 32a based on tbe converted control 
information. . . . - V . " 

[0090] At least the above-mentioned steps are exe- 
cuted by the computer. The control program may also 
include; for example; a control Information setting step, 
a moving body Meeting step r an invalid area setti ng 
step; amonitoring area setting step, a' designating step 
of designating the image area specified by each of these 
steps , an alarm information output control :step, and a 
zoom image olsplay step. 

[0091 ] Figure 4 is a perspective view illustrating a par- 
tial structure of the optical system 2a (Figure 3) of the 
omnidirectional camera 2, ■ 



[0092] In Figure 4 , . the optical system 2a includes a 
mirror of a convex body of revolution. When a hyperbolic 
curve is rotated about axis Z, a hyperboloid of two 
sheets is obtained The mirror of the convex body of rev- 
;5 .clution has a mirror surface oh a convex surface of one 
of the hyperboloids (2 > 0). 

[0093] The one hyperboloid is represented by the fol- 
lowing expressions. 
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{(X 2 +YVa 2 }Z 2 /b 2 = -1 
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(a 2 + b 2 ) 



[0094] • Here, "a" .and "b" are. constants defining the 
shape of the hyperboloid. "c M is a constant defining the 
position of the focal point of the hyperboloid. These ex- 
pressions and constants are included in conversion in- 

20 . formation for converting an omnidirectional image into 
a panoramic image or a perspective image. For exanv 
pie, these expressions and constants are. pre-stored in 
the program memory 3b in Figure 3. 
[0095] The mirror of a convex body of revolution has 

'25 two focal points, i.e., a first .focal point I and a second 
focal point ll. Dght entering from the outside of the . mir- 
ror and directed toward the first focal point I is reflected 
. by the optical system 2a (i :e. , by the mirror of the convex 
body of revolution) and is entirely directed to the second 

30 focal point II. v 

{0096] The rotation axis of the mirror of a convex body 
of revolution is matched to the optical axis of the camera 
leris.of the imaging section 2b, and a first principal point 
of the camera lens is located at the focal point ll.Owing 

35 to this arrangement, an image photographed by the im- 
aging section 2b has a center of view point at the first 
focal point 1. When the image is viewed with the first focal 
point I as the center of view, the position of the viewpoint 
is not changed in accordance with the viewing direction. 

40 [0097] . .Figure 5 shows Display Example (1) according 
to this example of the present invention. Display Exam- 
ple (1 ) is for setting control information. 
[0098] Figured shows a window 51 displayed on a 
display screen of the display section 3e. The window 51 

45 " snows a circular omnidirectional image : 52, and a pull- 
down menu 53. The pulldown menu 53 appears when 
the control information setting section 33a is started by 
ah input operation performed using a mouse or a key- 
board, the control information setting section 33a is 

so started for dividing the omnidirectional image 52 into a 
plurality of image areas and for adding an area number 
(preset number) to each of the divided image areas. 
[0099] . Figure 6 shows Display Example (2) according 
to this example of the present invention. Display Exam- 

55 pie (2) is for setting control information. " 

[0100] Figure 6 shows a window 61 displayed on the 
display screen of the display section 3e. the window 61 
shows a part of a circular omnidirectional image 62, and 
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a window 63. The window 63 appears when the control 
information setting section 33a is started by an input op- 
eration performed using a mouse or a -keyboard.. The 
control information setting section 33a is started for di- 
viding the-omnidirectional image $2 into a plurality -of 
image areas and for adding aaarea number (preset 
number)-to each of the divided image areas. Thewin- 
dow 63 allows the user to input numerical figures which 
represent how many image areas the omnidirectional 
image 62 is to be equally divided into concentrically and 
radially 

[0101] In this example, the omnidirectional image 62 
is divided into five concentric circles. The window 63 in 
Figure 6 shows the numbers of image areas included in 
the five concentric circles. Trie omnidirectional image 62 
after the division is shown in Figure ?: Here; the omni- 
directional image 62 is divided Into 255 image areas. : 
[0102] Figure 7 shows Display Example (3) according 
to this example of the present invention. Display Exam- 
ple (3) is for setting control information. 
[0103] Figure 7 shows a window 71 displayed on the 
display screen of the display section 3e. The window 71 
shows a circular omnidirectional image 72, .: image 
frames 73 obtained as a result of equally dividing the 
omnidirectional enage 72 concentrically and radially by 
the dividing section 31a based on the numerical figures 
which was input in the window 63(Figure 6), ;and area 
numbers 74 added to the divided Image areas respec- 
tively.. In .Figure 7,;only some of the -area numbers .are 
shown for the sake of simplicity, but actually area num- 
bers 1 through -255 are displayed in the respective irh- 
age areas: y: . - . : . 

[0104] : Control information for controlling the position 
and the magnification ratio of the zoom camera 1 is pro- 
vided i n correspondence with each of the divided image 
areas numbered 1 through 255. In this example, the om- 
nidirectional image 72 is-divided into 2551mage"areas, 
and the zoom camera 1 can be zoomed to any image 
area,.for example, the Image area numbered 2551: 
[01 05] . Figure 8 shows Display Example (4) according 
to this example of the present invention; Display Exam- 
ple (4) is for detecting a moving body, 
[0106] - Figure 8 shows a window 81 displayed on the 
display screen of the display section 3e. The window 81 
shows apart of an omnidirectional image 62 which Is 
mostly hidden, difference image data 83y a vertical cur- 
sor line 84, ahorizohtal cursor line 85, arid an intersec- 
tion 86 of the vertical and horizontal cursor lines 84 and 
85.The difference image data 83 (white parts) is ob- 
tained by performing pattern matching of successive 
frames of the omnidirectional image 82 which has been 
stored in the image memory section 3d: The difference 
image data 83 represents a part in . which the image-has 
been changed due to,, for example, a movement of a 
moving body. The vertical and horizontal cursor lines 84 
and 85 track the difference-data based on a certain cri- 
terion (for example, whether the area of the white parts 
is a prescribed level or higher), so that the intersection 



86 represents the position of the moving body. By auto- 
matically specifying an area including the position rep- 
resented by the intersection 86, the information on the 
image area including this position is converted into con- 
5 trol information for zoom^photographing this image area 
. by the zoom camera 1. . 
[0107] Figure 9 shows Display Example (5) according 
to this example of the present invention. Display Exam- 
ple <5) is for setting an invalid area 

10 [01 08] Figure 9 shows a window 91 displayed on the 
display screen of the display section 3e. The window 91 
shows a circular f omnidirectional' image 92, image 
frames 93 obtained as a result of dividing the omnidi- 
rectional image 92 equally by the dividing section 31a 

is based on the numerical figures which was input in the 
window 63 (Figure 6). and an image area 94 which has 
been preset as Invalid by the Invalid area setting section 
36a. As described above, the invalid area setting section 
36a presets, as invalid, an arbitrary image area in the 

20 omnidirectional image which is unstable due to the in- 
fluence of environment. The image area 94 preset as 
invalid by the invalid area setting section 36a is not sub- 
jected to the specification by the specifying section 32o 
or tho conversion by the converting section 34a. 

25 [0109] Figure 10 is a flowchart illustrating . a basic op- 
eration procedure for the composite camera system 10 
shown In Figure 1. 

[0110] As shown in Figure 10, an omnidirectional im- 
age is photographed in step S 1 . • M ore specifical ly, an 
30 omnidirectional image over a viewing angle of 360 de- 
grees around the omnidirectional camera 2 is taken by 
the omnidirectional camera 2. : ~ 
J0111] .In step S2, the obtained omnidirectional image 
is displayed on the display screen of the display section 

[0112] In step S3, it is determined whether or hot con- 
trol information for the omnidirectional image is to be 
set. When ihe control Information is to be set (YES), the 
. procedure goes to step S1 6; If thecontrol information is 

40 not to be set (NO),-the procedure goes to step S4. 
[0113] In step S4; it is determined whether or not an 
image-area in the omnidirectional image is to be zoom- 
photographed by automatic specification. When an im- 
age area in the omnidirectional Image is to be zoom- 

4s photographed by automatic specification (YES), the 
procedure goes to step S5. When no tnage area in the 
omnidirectional image is to be zoom-photographed by 
automatic specification; (NO), the procedure goes to 
step S15. <■ " - : - 

[0114] lnstepS5,itisdeterminedwhetherornotthere 
is an image area which has been preset as invalid by 
the invalid area setting section 36a: If there is such an 
area (YES), .the procedure goes to step S6. if there is 
no such area (NO), the procedure goes to step S7. 
ss [0115] in step S6, the omnidirectional image exclud- 
ing the image area preset as invalid is temporarily stored 
in" the image memory section 3d frame by frame sequen- 
tially. Then, the procedure goes to step S8. 
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{01 16] ..In step S7, since there is n o image area preset 
as invalicMhe entirety of the omnidirectional image is 
temporarily stored in the image memory section 3d 
frame by frame sequentially. 
[0117] Instep S8i itisdeterminedwhetherornotthere 
is difference image data in the omnidirectional image, 
using the moving body detecting section 35a. The dif- 
ference image data is image data which has been posi- 
tionally offset between two successive frames because 
of a movement of a moving body. If there is such data 
(YES), it Is determined that amoving body has been de- 
tected, «nd the procedure goes tcstep S9. If there Is no 
such data (NO), it . is determined that no moving body 
has been detected, and the procedure returns to step 
S6. In step S6; pattern matching is performed for suc- 
cessive frames of the omnidirectional image which have 
been stored in the image memory section 3d, thereby 
creating a difference image. The difference image rep- 
resents the positional offset of the. image data caused 
by a movement of a moving body. Based on the differ- 
ence image, it is determined whether or not there is a 
moving body. 

[0118] In step S9,;the specifying section 32a fe used 
to cause the cursor to track the imago data which Is po- 
sitionally offset by a prescribed level or greater,: in order 
to indicate mainly in which, image areas of the omnidi- 
rectional image that difference image data exists, Then, 
the area numbers of the image areas corresponding to 
the image data are sequentially specified. 
[0119] In step S 1 0, the convert ng section 34a is used 
to convert the information onthe images area specified 
by the specifying section;32a (for. example,,the area 
number of the image area) into the preset control infor- 
mation. ... . " i 
[01 20] In step S1 1 , the communication section 3h is 
used to send the converted control information to the 
zoom camera control unit 4 via one of the communica- 
tion lines 6. 

[0121] In step S12, the zoom camera unit 4 controls 
the position arid the magnification ratio of the zoom cam- 
era 1 based on the received control information via one 
of the communication lines 6. .: .; \. . ... 
[0122] In step S13, a target image area is zpom-pho- 
tographed by the zoom camera 1 . : : . 
[01 23] In step S1 4,. the zoom-photographed image Is 
displayed on the display screen of the monitor 5. Thus, 
the procedure is completed: . 

[0124] In step S15, when the omnidirectional image 
was not zoom-photographed (NO in step S4), the spec- 
ifying section 32a, which received an operation specifi- 
cation instruction through the 'mouse, is used to specify 
the area number of the image area to be zoom-photo- 
graphed by the zoom camera 1. Then, the procedure 
jumps to step S1 0. 

[0125] . In step S1 6, when the control information is to 
be set (YES instep S3) ; the dividing section 31a is used 
to divide the omnidirectional image displayed on the dis^ 
play section 3e concentrically and radially The area 



frame of each.of the divided image area in the omnidi- 
rectional image is displayed,, and the area number is 
added to each of the divided image areas. 
[0126] In step=S17,. after step S16, the control infor- 
5 mation setting section 33a is used to preset the control 
information for controlling the position and the magnifi- 
cation ratio of the zoom camera 1 for each image area 
provided with i the respective area number. 
[0127] :lnstepS18,afterstepS17,thepresetcontrol 
to information is stored In the control information table 3f. 
Then, the procedure goes to step S4. - 
[0128] As described above; the composite camera 
system 10 according to this example of the present in- 
vention includes the following: the zoom camera 1 ca- 
ts : pable of being driven to pan; tilt and zoom; an omnidi- 
rectional camera 2 including an optical system 2a for 
projecting an optical image over a viewing angle of a 
maximum of 360 degrees, and ah imaging section 2b 
for photographing the optical image as an omnidirec- 

20 tional im'age;:a display section 3e for displaying the pho- 
tographed omnidirectional Image; a dividing section 31a 
for dividing the displayed omnidirectional image into a 
plurality of image areas; a specifying section 32a for au- 
tomatically or manually specifying at least one image ar- 

& eafromthe plurality of Image areas; a converting section 
34a for converting the information on the image area 
specified by the -specifying section 32a into control in- 
formation for controlling the zoom camera i to be at a 
prescribed position and to have a prescribed magnifies- 

so tion ratio for. photographing-the specified image area; 
and a zoom camera control section 4 for controlling the 
position and the magnification ratio of the zoom camera 
.1 for photographing thespecrfied image area, based on 
the converted comrbl . information. . Owing to such a 

35 structure; the composite camera system 10 can take an 
omnidirectional image including images of a plurality of 
important sites using the omnidirectional camera 2, and 
display the images of these important sites simultane- 
ously as included in the: omnidirectional image. The 

40 composite camera system 10 can also zoom-photo- 
graph a desired, specified imagearea In the omnidirec- 
tionalimage using the zoom.camera 1. . 
[0129] in theabove example, the remote controller 3 
is connected to the ^omnidirectional camera 2 and the 

4* zoom camera 1 via the wireless or wired communication 
lines.6; Alternatively, the remote controller 3 and the 
zoom camera control unit 4 may be combined into one 
unit, orihe remote controller'3 may be provided in the 
vicinity of the omnidirectional camera 2 and the zoom 

so camera 1 as a controller. 

[0130] ln\the above example,. the composite camera 
system 10 includes one zoom camera 2, Tho present 
.invention is not fimited to such a structure. The control 
section 3a of the remote controller 3 may control the po- 

55 sition and the magnification ratio of a plurality of zoom 
cameras via the zoom camera control unit .4. In this 
case , a plurality of display screens of the display section 
3e may be provided in correspondence with the plurality 
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of zoom cameras, or one display screen may display 
Images corresponding to the plurality of zoom cameras 
sequentially. . -. ■ 

[0131] In the above example, the position of the zoom 
camera is controfled by pah driving and tilt driving/and 
the magnification of the zoom camera is controlled by 
zoom. driving. The pan driving,. the. tilt driving-and the 
zoom driving are merely examples for controlling the po- 
sition and magnification ratio; and the present invention 
is not limited to this. . 

[0132] As described above, according to the present 
invention, positional control and magnification ratio con- 
trol of the zoorri camera are performed centered on a 
prescribed portion of omnidirectional image data, and 
the omnidirectional image and the zoom image are dis- 
played. The omnidirectional camera is used to photo- 
graph a plurality of important locations simultaneously 
as Included in the omnidirectional image, and the zoom 
camera is used to zoom-photograph a specified image 
area in the omnidirectional image; By specifying a pre- 
scribed image area in the omnidirectional image, the po- 
sition of the.zoom camera can toe easily set. 
[0133] The omnidirectional image over a viewing an- 
gle of a maximum of 360 degrees taken by the omnidi- 
rectional camera is divided into a plurality of image ar- 
eas, and an image area of interest is automatically or 
manually specified among the plurality of image areas. 
Thus; : the specified image area of interest is zodm-pho- 
tographed by the zoom camera which can-be driven to 
pan; tilt and zoom. Since a large-area monitoring area 
is photographed by one omnidirectional camera, it is not 
necessary to install a plurality of monitoring sensors, for 
example,, a plurality of detecting sensors or input sen- 
sors forautpmaticspecificationblthe target Image area. 
[0134] The plurality of image areas obtained by divid- 
ing the omnidirectional image act as a plurality of detec- 
tion sensors or input sensors to calculate the control in- 
formation for the image area automatically or manually 
specified. Based on the control information, the zoom 
camera is controlled to 'be at a prescribed position (to 
assume a prescribed photographing direction)^ and to 
have a prescribed magnificatiori ratio: : \\ 
[0135] The zoom camera can track a moving body in- 
cluded in the omnidirectional image taken by the omni- 
directional camera, . zoom-photograph the moving body, 
and display an enlarged image of the moving body on 
a display screen of the display section. . 
[0136] In the case where a circular omnidirectional im- 
age is divided into a plurality of image areas; arid control 
information is set for controlling the zoorh camera to be 
at a prescribed position and to have a prescribed mag- 
nification ratio for each of the plurality of image areas, 
the division can be performed effectively and the image 
areas can be displayed so as to be easily identifiable. 
[0137]' Control information for controlling the zoom 
camera to be at a prescribed posftion and to have a pre- 
scribed magnification ratio for each of the plurality of im- 
age areas provided with the respective area numbers 



can be preset.while viewing the omnidirectional image 
displayed on the display screen of the display section. 
[0138] , An image area, fin the omnidirectional image, 
which is unstable due to environmental influence can be 

5 invalidated in advance: Therefore, a moving body or an 
abnormality can be accurately detected automatically 
from trie omnidirectional image, so that the position and 
the magnification ratio of the zoom camera is accurately 
controlled for the detected moving body or the site of the 

10 abnormality. ■ \ - . 

[0139] A plurality of monitoring areas which are to be 
photographed in different photographing directions can 
be set for one omnidirectional camera. -When a moving 
body is detected to have entered the monitoring area, 

is alarm information including an alarm sound is output 
and also the zoom camera is zoomed to the moving 
body to display the image of the moving body on the 
display screen of the display section: 
-[0140] The omnidirectional camera and the zoom 

20 camera can be remote-controlled via a communication 
line. Therefore; at a site where a computer is installed, 
the above-described division of the omnidirectional im- 
age, preset of the control information for controlling the 
position and the magnification ratio of the zoom camera 

25 fo r a specified image area; arid photographing and dis- 
play of an area including the specified image area can 
be performed. 

[0141] Various other modifications will beapparentto 
andean be readily made by those skilled intheartwith- 
30 out departing from the scope and spirit of this invention. 
: Accordingly,\it 1st not intended that the -scope of the 
.claims appended hereto be limited to the description as 
set forth herein, but rather that the claims be broadly 
construed. 

Claims ,: 

1 ; A composite camera system, comprising: 
40 . - -. -•'- •/ • ■ 

a control section for performing positional con- 
trol and magnification ratio control of at least 
.-: one zoom camera ratio for omnidirectional im- 
age data centered on a prescribed portion 
'45 .: ; thereof, the omnidirectional Image data being 
. obtained by an omnidirectional camera capable 
of taking an omnidirectional image over a view- 
ing angle of a maximum of 360 degrees; and 
a display section fordisplaying an omnidirec- 
so - tional image taken by the omnidirectional cam- 

era and a zoom image taken by the zoom cam- 
- - era. ' ; ' ' - 

2. A composite camera system according to claim 1 , 
55 wherein the positional control for controlling a pho- 
- tographic position is performed by pari driving and 
\ tilt driving; and the magnification ratio control is per- 
forrried by zoom driving. 
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3. A composite camera system according to claim 1 , 
wherein the prescribed portion of the omnidirection- 
al image data is a divided image area which is set 
in a photographed image. 

4. A composite camera system according to claim 1 , 
wherein the control section includes: 

a dividing section for dividingthe omnidirection- 
al image displayed on adisplay screen of the 
display section into a plurality of image areas; 
a specifying section for automatically or manu- 
.... airy specifying at least one image area from the 
plurality of image areas; 
a converting section for converting information 
, on the at least one image area specified by the 
. . specifying section into control information for 
controlling the zoom camera :to be at a pre- 
scribed position and to have a prescribed mag- 
nification ratio for photographing an image area 
specified by the specifying section; and 
a zoom camera control section for controlling 
the zoom camera to be at the prescribed posi- 
tion and to have the prescribed magnification 
ratio for photographing the Image area speci- 
fied by . the specifying section, .based oh the 
control information converted by the converting 
section. 

5. A composite camera system according to claim 1, 
wherein the omnidirectional camera includes an op- 
tical system for projecting the omnidirectional im- 
age using a mirror of a parabolic or hyperbolic con- 
. vex body of revolution, or a fish-eye lens. 

6. A composite camera system according to claim 4, 
further comprising: , . . 

■ a control information setting section for preset- 
ting the control information for controlling the 
zoom camera to be at the prescribed position 
and to have the prescribed magnification ratio 
for each of the plurality of image areas obtained 
by the dividing'section; and 
a control information table for storing the preset 
. control information in correspondence with the 
position of each of the plurality of image areas, 

wherein the converting section obtains :the 
control information for the image area specified by 
the specifying section from the control information 
table and/or from calculation using a prescribed cal- 
culation procedure. . 

7. A composite camera system according to claim 4, 
further comprising: 

an image memory section for temporarily stor- 
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. ing the omnidirectional image taken by the om- 

. nidirectiorial camera frame by frame sequen- 

• tially;and. 
a moving body-detection section for detecting 
a .movement of a moving body by performing 

; pattern matching of successive frames of the 
. . omnidirectional image stored in the image 
memory section; . 

wherein the detected moving body is zoom- 
photographed by the zoom: camera, and an en- 
larged image of the moving body is displayed on the 
display screen of the display section. 

A composite camera system according to claim 4, 
wherein when the omnidirectional image is circular, 
the dividing section divides the omnidirectional im- 
age concentrically and radially, and displays the 
: plurality of image areas of the omnidirectional im- 
age together with area frames defining the plurality 
of image areas.: 
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9. - . A composite camera system according to claim 6, 

wherein the dividing section displays the plurality of 
image areas of the omnidirectional irhage together 

. with area frames and area numbers defining the 

v plurality of image areas,rand the control inf ormation 
setting section presets the control information for 
controlling the zoom camera to be at the prescribed 

. position and to : have the prescribed magnification 
ratio for each of the plurality of image areas provid- 

.;; ,ed with the area numbers. . 

10. A composite camera system according to claim 4, 
; furtherpomprising an invalid area setting section for 
. . presetting, as invalid, an arbitrary area of the omni- 
. . directionalimagewhich is unstable due to environ- 
mental influence, 

wherein the invalid area setting section inval- 
idates- the specification, of the' image area by the 
specifying section. 

11. A composite-camera system according to claim 7, 
further comprising: 

a monitoring area setting section for setting at 
least one monitoring area in the omnidirectional 
. image; and 

an alarm information output section for output- 
. ting alarm information when the moving body 
■v.. detecting section detects thatthe moving body 
. has entered the monitoring area 

wherein the zoom camera zoom-photographs 
the moving body which has entered the monitoring 
area and displays the. photographed image of the 
. moving body on the display screen of the display 
-section. - 
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12. A composite camera system according to claim 1 , 
further comprising a remote controller connected to 
the omnidirectional camera and'the zoom camera 
via a wireless or wired communication line, 

. ■ .wherein an omnidirectionat imapje' taken by 
the omnidirectional camera and an image taken by 
the zoom camera are transferred to the remote con- 
troller via the communication line. 

13. A composite camera .system according to claim 1 2, 
.. wherein the remote controller includes at least the 

dividing section, the specifying section, and the 
converting section,. 

14. A composite camera image display control method, 
. comprising the processing steps'of : , 

performing positional control and magnification 
ratio control of at least one zoom camera for 
omnidirectional image data centered on a pre- 
scribed portion thereof, the omnidirectional im- 
age data being obtained by ah omnidirectional 
. - camera . capable: of taking an. omnidirectional 
Jmago.over a viewing angle of a maxirhum of 
- 360 degrees; and 

displaying an omnidirectional image taken by 
the omnidirectional camera and a zoom image 
. . . - taken by the zoom camera. 

15. A zoom camera control method for performing po- 
. sitional control and magnification ratio control of at 

least one zoom camera.for omnidirectional image . 
data centered on a prescribed portion thereof, the 
. omnidirectional image data' being obtained by an 
. omnidirectional camera capable of taking an omni- 
directional image over a viewing angle'of a maxi- 
mum of 360 degrees, the method comprising the 
steps of: 

displaying an omnidirectional image taken by 
the omnidirectional camera; , , 
dividing the omnidirectional image displayed on 
a display screen of the display section into a 
plurality of image areas; . 
automatically or manually specifying at least 
. . - one Image area from the plurality of image ar- 

; eas;. ;-r\/ - c- \ -'. 

converting information oh the.«t least one im- 
age area specified by the specifying section in- 
to control information for controlling the zoom 
camera to be at a prescribed position and to 
have a prescribed magnification ratio for pho- . 
tographing the image area specified by the 
specifying section; and . 

. .■■ controlling :the zoom camera to be at the pre- 
scribed position and to. have the prescribed 
■ magnification ratio for photographing the image 
area specified by the specifying section, based 
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on the control information converted by the con- 
. : . verting section. 

16. A control program for causing a computer to exe- 
cute the steps of : 

displaying an omnidirectional inhage taken by 
the omnidirectional camera; 
.dividing the omnidirectional image displayed on 
a display screen of the display section into a 
. plurality of image areas; •' 

automatically or manually specifying at least 
one image arek from the plurality of image ar- 
eas; 

- converting information on the at least one im- 
age area specified by the specifying section in- 
to control Information for controlling the zoom 
camera to be at a prescribed position and to 
have a prescribed magnification ratio for pho- 

: rtographing the image area specified by the 
specifying section; and 

controlling the zoom camera to be at the pre- 
scribed position and to have the prescribed 
magnification ratio for photographing the imago 
area specified by the specifying section, based 
oh the control information converted by the con- 
verting section. 

17. A computer readable recording medium having a 
■ control program according to claim 1 6 stored there- 

•: in/: : ■.<: • - . - . 

1B. A control program for causing a computer to exe- 
cute the processing steps of a composite camera 
image display control method according to claim 14. 

19. A computer readable recording medium having a 
control program according to claim 1 8 stored there- 

.- in.. , ■ ■: 
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